1. Technical Field
This invention generally relates to computer systems, and more specifically relates to testing of computer systems.
2. Background Art
Modern life is becoming more dependent upon computers. Computers have evolved into extremely sophisticated devices, and may be found in many different applications. These applications involve everything from application specific computers found in everyday devices such as automobiles, phones and other electronics, to the general purpose computers found in the form of PDAs, personal computers, servers and mainframes.
As computers become more integrated into daily life, their reliability becomes a greater and greater necessity. In order to ensure sufficient reliability it is necessary to thoroughly test computer systems. Thorough testing involves testing both the hardware and software of the computing system to ensure that the system operates properly in a wide range of situations.
One of the more difficult areas in computer system performance to test is the computer system's response to errors in operation. For example, it has been especially difficult to test built-in test routines for response to errors in those routines. The major difficulty in testing a computer's response to errors is in steps that need to be taken to inject the errors into the system. Typically, this has been accomplished with intrusive methods.
For example, a debugger or emulator is used to set breakpoints into the software at which the errors are injected by operator command. These methods are tedious, time consuming and not easily automated. Another method is to instrument the code to cause the error. This involves the creation of a special version of the software that is hard coded to cause the error. This approach raises several issues. The first being that multiple versions of the software must be maintained. The second issue is that the software being tested is no longer the actual operational software, and the actual operational software may in fact respond differently then the test software.
Additionally, it can be especially difficult to inject errors that represent hardware failures. Thus, it can be extremely difficult to evaluate software's response to hardware errors.
For these reasons, the computer system's response to errors may not be tested fully. Instead, only a few manual tests may be performed, and they may not be repeated as the application is modified in the future. Or in some cases, the failure conditions may not be tested at all.
Thus, what is needed is an improved testing system and method that provides for more complete testing of a computer system's response to errors.